Closed Bug 1208415 Opened 9 years ago Closed 8 years ago

The H264 DXVA missing frame detector is way too sensitive

Categories

(Core :: Audio/Video: Playback, defect, P5)

Product:
Core
Component:
Audio/Video: Playback
Version:
42 Branch
Platform:
x86_64
Windows 7
Type:
defect

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1257013

People

(Reporter: clogged.drainpipe, Unassigned)

Details

Attachments

(2 files)

less sensitive video-decoding-fallback MediaDecoderStateMachine.cpp
100.34 KB, patch
Details | Diff | Splinter Review
Make corrupt frame detector be adjustable
4.03 KB, patch
cpearce
: review-
Details | Diff | Splinter Review 
As of FF44a1(Nightly 2015-09-24) it is very easy to trigger the framedrop detector and get DXVA disabled, by seeking around randomly in a 1080p30 youtube video. This should not happen and quite annoying.
I think the framedrop detector should be less sensitive, and I would love to see a pref to disable it completely.
Component: Untriaged → Audio/Video
Product: Firefox → Core
Version: 44 Branch → 42 Branch
I believe this affects all current and planned versions. (40,41,42,43,44) Tested on 41,42 and 44.
Older versions might also be affected, possibly well into the 3x series.
ATI HD4890
It is intended to detect failures in exactly that class of GPU. I've been testing the HD4600 which is where the GPU failure is easy to reproduce. Is the transition from hardware decoding to software decoding looking too lumpy?

You can disable hardware decoding altogether by setting media.hardware-video-decoding.enabled=false
I know that the HD4XXX series is very troublesome in terms of DXVA, but for me, on my system it can work without serious issues. (I have done extensive testing, framedrops seem to happen in a short burst when seeking, but after that, there are no issues)

So I am annoyed because I want FF to use DXVA (for better or worse), but there is no option to force it.
Priority: -- → P5
I'd accept a patch if you want to fix it.
I would, but I have never worked on Firefox before, so I do not have a build environment or the dependencies set up, and looking at the MDN guide it seems to be quite the hassle to get it going.
If it was only the coding I would probably get around doing it, but in the foreseeable future I doubt I will have the time and patience to set up the build env.

I have been looking at the source code though, and found the culprit in MediaDecoderStateMachine.cpp:
if (mReader->VideoIsHardwareAccelerated() &&
    frameStats.GetPresentedFrames() > 60 &&
    mCorruptFrames.mean() >= 2 /* 20% */)

Adding an additional boolean (with value taken from a pref) in this if statement would disable the corrupt frame detector.
Yes. It is simple and I know how to do it. IIRC I reviewed that patch. Which is why I suggested you give it a go if you feel it is important.

My experience is pretty terrible and as you can see it is triggered if more than 20% of frames aren't being delivered to the screen for some reason better known to Microsoft or AMD. If you crank up the frame rate and resolution you can get to the point where the GPU doesn't deliver any frames at all.

When we originally discovered this problem we were showing black frames rather than dropping them. I could get it to be completely black for several seconds. I'm not convinced that anyone wants that but you seem to think otherwise, so you're welcome to experiment on your own time.

I've already spent several days on this issue working with Matt Woodrow. The current situation is as good as we think we can get without having the underlying issue fixed in the driver. We have worked really hard to get the AMD cards enabled for hardware acceleration as much as possible but ultimately they're a bit dudsey.
Summary: The H264 DXVA dropped/corrupted frame detector is way too sensitive → The H264 DXVA missing frame detector is way too sensitive
I understand your position, and that you do not want to spend any more effort on this issue.
Anyways, thanks for all the work you guys have done so far, I am sure that it improved user experience for a lot of people.
When I get around to making a patch, I will post it here.
Severity: normal → minor
Component: Audio/Video → Audio/Video: Playback
Please add an option to completely disable this fallback.
(In reply to tomy_homy from comment #10)
> Please add an option to completely disable this fallback.

See c6 above.
I think the following change should not enable the fallback too fast - at the beginning of video playback. Also should not trigger the fallback when we skip some part of videos.

if (mReader->VideoIsHardwareAccelerated() &&
    frameStats.GetPresentedFrames() > 300 &&
    mCorruptFrames.mean() >= 6 /* 60% */)
media.hardware-video-decoding.force-enabled;true
does not prevent triggering video decoding fallback which sucks. Same for
gfx.blocklist.all;-1
Latest FF build that does not trigger decoding fallback is Firefox 37.0.2 x86
less sensitive video-decoding-fallback in MediaDecoderStateMachine.cpp as per comment12 https://bugzilla.mozilla.org/show_bug.cgi?id=1208415#c12
Flags: needinfo?(clogged.drainpipe)
Attachment #8714081 - Flags: review?(ajones)
// Comments around the fallback function were outdated and I forgot to update them as well

A little bit of explanation why such values:
I use low-end AMD E-450 APU with HD6320 graphics which supports H264 hardware decoding, 16.1 AMD drivers and Windows 7 x64 SP1.

The video decoding fallback was not yet implemented in Firefox 37.0.2 so 1080p@30fps videos play smoothly using the outdated version.

Firefox >37.0.2 already has this the most ridiculous function I have ever seen which is triggered in ~80% of cases when I move playback forwards/backwards. 

1) The APU needs couple (3-7) seconds to finish loading YT/other webpage with video. Also it needs ~3 more seconds to change video quality to 1080p in case of user did it manually, via extension, userscript OR just clicked fullscreen button. So the fallback should not be triggered for first 10 seconds of the video playback.

2) Also every single time I move video playback forwards/backwards, the APU needs up to 3-5 seconds to drop CPU usage to normal (like when in smooth playback state). The 3-5 seconds skipping video playback lag is enough to trigger video decoding fallback on Firefox >37.0.2. Website or if part of video I skip is already buffered or not does not matter AS MUCH as you may think.

Of course those times described here are correct assuming there is no heavy task running in background. Automatic Windows Update updates check, anti-virus database updade etc. could still trigger the fallback. That's why I chose to set the fallback to trigger if >60% of latest 300 frames has been corrupted.

Seriously, not everyone has the newest CPU/GPU available, but it does NOT mean it's so bad it can't handle 1080p@30fps. Let's just see how it works, please.


Also using 37.0.2, 16.1 AMD drivers and playing 1080p@30fps videos I haven't noticed any black frames or other artifacts. I would suggest to completely remove the fallback function and build test build to check for any possible issues (doubt it there are any for me in this case).
I do not have the dev environment set up , so I cant compile FF with your changes, but thanks for pursuing this, I will test it when a binary is available.
Flags: needinfo?(clogged.drainpipe)
Me too. I also don't have access to 'Try Server'. https://wiki.mozilla.org/ReleaseEngineering/TryServer#Try_Server

Dunno why the attachment diff does not work. I did 3 changes:
1) from
> mCorruptFrames(30),
to
> mCorruptFrames(300),
2) from
> frameStats.GetPresentedFrames() > 60 &&
to
> frameStats.GetPresentedFrames() > 300 &&
3) from
> mCorruptFrames.mean() >= 2 /* 20% */) {
to
> mCorruptFrames.mean() >= 6 /* 60% */) {
Comment on attachment 8714248 [details] [diff] [review]
Make corrupt frame detector be adjustable

Review of attachment 8714248 [details] [diff] [review]:
-----------------------------------------------------------------

There are only two variables here. The number of frames and the threshold percentage. Perhaps they could be called media.corrupt-frames.count and media.corrupt-frames.threshold. mCorruptFramesDetail is not consistent with out naming conventions. I'll defer to Chris for other details such as when/how we read the prefs.
Attachment #8714248 - Flags: review?(cpearce)
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #19)
> Comment on attachment 8714248 [details] [diff] [review]
> Make corrupt frame detector be adjustable
> 
> Review of attachment 8714248 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> There are only two variables here. The number of frames and the threshold
> percentage. Perhaps they could be called media.corrupt-frames.count and
> media.corrupt-frames.threshold. mCorruptFramesDetail is not consistent with
> out naming conventions. I'll defer to Chris for other details such as
> when/how we read the prefs.

I don't know how to read the prefs only once. GFX can do that, but how can DOM module do it?
This will reduce the prefs reading overhead, to avoid the increasing higher dropped frames.
Comment on attachment 8714248 [details] [diff] [review]
Make corrupt frame detector be adjustable

Review of attachment 8714248 [details] [diff] [review]:
-----------------------------------------------------------------

Thanks for the patch. Please make the changes below, resubmit for review, and we can get this landed.

And I disagree with Anthony, I think it makes sense to have all three variables as prefs.

::: dom/media/MediaDecoderStateMachine.cpp
@@ +168,5 @@
>  
> +namespace mCorruptFramesDetail {
> +
> +uint32_t mCorruptFramesInit() {
> +  uint32_t temp = mozilla::Preferences::GetInt("mcorruptframes.init", 60);

This is not how we normally do prefs... See below...

@@ +246,5 @@
>    mDropAudioUntilNextDiscontinuity(false),
>    mDropVideoUntilNextDiscontinuity(false),
>    mDecodeToSeekTarget(false),
>    mCurrentTimeBeforeSeek(0),
> +  mCorruptFrames(mCorruptFramesDetail::mCorruptFramesInit()),

Please add data fields to the MediaDecoderStateMachine class to cache the pref values, like so:

mCorruptFrames(Preferences::GetInt("media.corrupt-frames.window-length", 60)),
mCorruptFramesPresentedThreshold(Preferences::GetInt("media.corrupt-frames.presented-threshold", 60)),
mCorruptFramesMeanThreshold(Preferences::GetInt("media.corrupt-frames.threshold", 2)),

Note this object is created every time you load a new video. So for changes to the prefs to cause a behaviour change, you need to reload the page.
Attachment #8714248 - Flags: review?(cpearce) → review-
(In reply to Chris Pearce (:cpearce) from comment #21)
> And I disagree with Anthony, I think it makes sense to have all three
> variables as prefs.

I don't think the rolling average window and the number of frames before we check the rolling average are in fact separate variables. Matt - correct me if I'm wrong.
Flags: needinfo?(matt.woodrow)
(In reply to Anthony Jones (:kentuckyfriedtakahe, :k17e) from comment #22)
> (In reply to Chris Pearce (:cpearce) from comment #21)
> > And I disagree with Anthony, I think it makes sense to have all three
> > variables as prefs.
> 
> I don't think the rolling average window and the number of frames before we
> check the rolling average are in fact separate variables. Matt - correct me
> if I'm wrong.

They are separate, one is specified in the constructor and the other each time we try check for corrupt frames.
Flags: needinfo?(matt.woodrow)
The video decoding fallback should never exist. Only someone who works for Google could approve it.
(In reply to tomy_homy from comment #24)
> The video decoding fallback should never exist. Only someone who works for
> Google could approve it.

You mean falling back to software decoding if HW accelerated decoding seems to be failing, returning garbage, completely black frames, etc? This is a very much needed feature, there are a lot of buggy drivers and broken installations out there where this is necessary.

The only thing that merits discussion is how to make it work reliably, and giving users the freedom to disable it at their own risk.
"Very much needed feature" Seriously? If someone has broken installation or is using buggy drivers, he should fix HIS problems, install newer driver or reinstall FF.
Black frames shows up when the function is being triggered (at the time FF switches decoding).

God NO! This function in current state affects EVERYONE, just not everyone knows about it OR some has decent CPU with proper hardware acceleration. If it affect everyone then this is ridiculous. No one with right mind would implement this that way.

"how to make it work reliably" Make it not working at all OR allow user to choose if to disable HW video decoding OR not, if there are a lot of corrupted frames for at least few latest seconds.

You're putting newbie with bugged drivers/installation to the same bag with "PRO" users. Software decoding is not ANY better than HW (if CPU can't handle it)

I have E-450 which can handle 1080p@30fps using FF 37.0.2. The ridiculous function was not yet implemented, so I can safely skip video playback forwards/backwards without issues. Of course this APU is slow, but fast enough.
With more recent FF versions even if I won't do anything while playing 1080p@30fps, the ridiculous video decoding fallback is still being triggered for no reason.  Moving video playback forwards/backwards triggers the function pretty much all the time for me.

FF 37.0.2 x86 -> no ridiculous function = fine video playback
FF >37.0.2 -> ridiculous function = 720@30fps impossible to watch
Surprise me, tell me there is a about:config settings to disable frames relating statistics, please. No stats = no ridiculous functions.
(In reply to tomy_homy from comment #26)
> FF 37.0.2 x86 -> no ridiculous function = fine video playback
> FF >37.0.2 -> ridiculous function = 720@30fps impossible to watch

You are being unreasonable and mildly insulting. The underlying issue is a buggy platform. We're doing our best to work around it. We are doing our best to make video playback work on broken machines but it turns out not to be as straightforward as you think.
It's because:
1) you're trying to workaround buggy platform by ruining video playback on all platforms
2) you implement something without an option to force disable it
3) it's one of the reasons why you can see FF market share goes down
4) you're forcing me to build complicated environment and waste time to re-compile FF just to delete FEW lines of ridiculous code
5) not even talking about any patch you will not accept
and so on...

I can't believe this BUG still has unconfirmed status
(In reply to tomy_homy from comment #29)
> It's because:

None of this prevents you being polite.

Can you try media.windows-media-foundation.allow-d3d11-dxva=true to see if that helps?
(In reply to tomy_homy from comment #26)
> If someone has broken installation or
> is using buggy drivers, he should fix HIS problems, install newer driver or
> reinstall FF.

I am sorry to be the one to break it to you but 95% of people are not like us, they do not even know what a driver is, let alone how to update it. It might sound ridiculous at first, but we live in a world full of noobs, dumbasses and people who are both ignorant and unwilling to learn technical stuff.
This is why we see a great number of Apple **** being sold, a lot of people want things to "just work".
Telling technologically inept people to update their drivers is not very effective.

> Black frames shows up when the function is being triggered (at the time FF
> switches decoding).

Not true, some AMD cards with some drivers return with black frames, see https://bugzilla.mozilla.org/show_bug.cgi?id=1120128

> God NO! This function in current state affects EVERYONE, just not everyone
> knows about it OR some has decent CPU with proper hardware acceleration. If
> it affect everyone then this is ridiculous. No one with right mind would
> implement this that way.

I agree that this implementation is a loose cannon. But, again it means it should be FIXED.
I'm sorry, I didn't want to be unkind. This issue is driving me crazy.

media.windows-media-foundation.allow-d3d11-dxva=true
works unstable. It gives me couple dropped frames once per few seconds. One frame freezes for ~0,5 sec, then some previous frames are shown, then video plays forward again. If I move playback forwards, video decoding fallback is activated immediately.

Yes, that's sad, but those 95% should not limit rest of people.

One year old thread :P

If I just knew how to attach patched properly...
(In reply to tomy_homy from comment #32)

> media.windows-media-foundation.allow-d3d11-dxva=true
> works unstable. It gives me couple dropped frames once per few seconds. One
> frame freezes for ~0,5 sec, then some previous frames are shown, then video
> plays forward again. If I move playback forwards, video decoding fallback is
> activated immediately.

This is very confusing, the d3d11 code doesn't integrate with the fallback, so this shouldn't be possible.
(In reply to Matt Woodrow (:mattwoodrow) from comment #33)
> (In reply to tomy_homy from comment #32)
> 
> > media.windows-media-foundation.allow-d3d11-dxva=true
> > works unstable. It gives me couple dropped frames once per few seconds. One
> > frame freezes for ~0,5 sec, then some previous frames are shown, then video
> > plays forward again. If I move playback forwards, video decoding fallback is
> > activated immediately.
> 
> This is very confusing, the d3d11 code doesn't integrate with the fallback,
> so this shouldn't be possible.
tomy_homy is using Windows 7 according to comment 15. I would expect things to be horribly broken for him with the D3D11 pref turned on since that only works on Windows 8+.
The pref will have no effect on Win 7, so the worse performance he's seeing must be unrelated.

I'm going to see if I can get the D3D11 code working on Windows 7, that might be a nice solution to this.
Matt, I just described what I've noticed. I don't know the code as well as you guys.
Yes, I'm using Windows 7 x64 and FF 44.0.2 x64 as well as 'portable' FF 37.0.2 x86 whenever I want to watch videos.
Outside of Sandboxie cache is empty. Both versions run in separate sandboxes, so settings does not overwritten.
Tony, would you be willing to give this build a try:

http://archive.mozilla.org/pub/firefox/try-builds/mwoodrow@mozilla.com-49a2f516b9b11480a11c2f11752999b49a873e06/try-win32/

Both with and without the d3d11 pref set (and check about:support for h264 decoding, it should now say if you're on d3d9 or d3d11 for h264).
*Tomy_homy, sorry.
Matt, thank you, I appreciate this. Unfortunately my ADSL2+ line is completely broken for a week and my mobile connection is 512/128 kb/s, with reconnecting once per an hour, so this build need to wait... a while.
(In reply to Matt Woodrow (:mattwoodrow) from comment #37)
> Tony, would you be willing to give this build a try:
> 
> http://archive.mozilla.org/pub/firefox/try-builds/mwoodrow@mozilla.com-
> 49a2f516b9b11480a11c2f11752999b49a873e06/try-win32/
> 
> Both with and without the d3d11 pref set (and check about:support for h264
> decoding, it should now say if you're on d3d9 or d3d11 for h264).

For me, with d3d11 it ends up spitting this out under Supports Hardware H264 Decoding: "No; MFCreateDXGIDeviceManager failed with code 80004005"

This is with an AMD Radeon HD 6620G, Win7 SP1 x64, and the latest drivers (16.1.1)
(In reply to Kyle Repinski from comment #40)
> (In reply to Matt Woodrow (:mattwoodrow) from comment #37)
> > Tony, would you be willing to give this build a try:
> > 
> > http://archive.mozilla.org/pub/firefox/try-builds/mwoodrow@mozilla.com-
> > 49a2f516b9b11480a11c2f11752999b49a873e06/try-win32/
> > 
> > Both with and without the d3d11 pref set (and check about:support for h264
> > decoding, it should now say if you're on d3d9 or d3d11 for h264).
> 
> For me, with d3d11 it ends up spitting this out under Supports Hardware H264
> Decoding: "No; MFCreateDXGIDeviceManager failed with code 80004005"
> 
> This is with an AMD Radeon HD 6620G, Win7 SP1 x64, and the latest drivers
> (16.1.1)

According to msdn (https://msdn.microsoft.com/en-us/library/windows/desktop/hh162750%28v=vs.85%29.aspx) this function needs Win8.
(In reply to clogged.drainpipe from comment #41)
> (In reply to Kyle Repinski from comment #40)
> > (In reply to Matt Woodrow (:mattwoodrow) from comment #37)

> 
> According to msdn
> (https://msdn.microsoft.com/en-us/library/windows/desktop/hh162750%28v=vs.
> 85%29.aspx) this function needs Win8.

MSDN lies.

This build should work: http://archive.mozilla.org/pub/firefox/try-builds/mwoodrow@mozilla.com-3e859bd0bfedad8efb413ae54f1e56d0d124f923/try-win32/

Apparently it doesn't perform well though, so I need to think about it more.
(In reply to Matt Woodrow (:mattwoodrow) from comment #42)
> (In reply to clogged.drainpipe from comment #41)
> > (In reply to Kyle Repinski from comment #40)
> > > (In reply to Matt Woodrow (:mattwoodrow) from comment #37)
> 
> > 
> > According to msdn
> > (https://msdn.microsoft.com/en-us/library/windows/desktop/hh162750%28v=vs.
> > 85%29.aspx) this function needs Win8.
> 
> MSDN lies.
> 
> This build should work:
> http://archive.mozilla.org/pub/firefox/try-builds/mwoodrow@mozilla.com-
> 3e859bd0bfedad8efb413ae54f1e56d0d124f923/try-win32/
> 
> Apparently it doesn't perform well though, so I need to think about it more.

To say it doesn't perform well is a bit of an understatement unfortunately. At best it's showing maybe every 10th frame or so. Tried ruling a few things out with that build in case it helps:
1. It's bad regardless of the video's framerate, resolution, and bitrate.
2. Setting my display resolution to 1280x720 instead of 1366x768 to avoid any scaling did not improve it.
3. D3D9 DXVA still works well.

Can you by chance explain what's so different with the requirements for Win7 D3D11 DXVA? If I could get a little background here maybe I could try tinkering with it.
(In reply to Kyle Repinski from comment #43) 
> To say it doesn't perform well is a bit of an understatement unfortunately.
> At best it's showing maybe every 10th frame or so. Tried ruling a few things
> out with that build in case it helps:
> 1. It's bad regardless of the video's framerate, resolution, and bitrate.
> 2. Setting my display resolution to 1280x720 instead of 1366x768 to avoid
> any scaling did not improve it.
> 3. D3D9 DXVA still works well.
> 
> Can you by chance explain what's so different with the requirements for Win7
> D3D11 DXVA? If I could get a little background here maybe I could try
> tinkering with it.

Ok, performs awfully :) It does work reasonably well on my windows 10 machine though.

What differences do you mean exactly?

The Win7 specific problem is that the Windows Platform Update for Windows7 backported all the d3d11 Video APIs, but it didn't backport the 'CLSID_VideoProcessorMFT' MediaFoundation Transform that we use for converting YUV->RGB when using D3D11 DXVA.

This MediaFoundation Transform is basically just a convenience wrapper though (albeit a closed source one), so the above try builds are where I reimplemented an equivalent using the ID3D11VideoProcessor etc APIs directly.

It's possible that I did it wrong, or am passing certain parameters that cause it to take a slow path, but I don't see anything obvious.

One interesting thing to test would be to skip the colour conversion entirely, and just output dummy frames instead, something like this in CopyToImage:

RefPtr<SourceSurface> surf = Factory::CreateDataSourceSurface(IntSize(1,1), SurfaceFormat::B8G8R8X8, false); 
RefPtr<layers::image> image = new SourceSurfaceImage(IntSize(1,1), surf);
image.forget(aOutImage); 
return S_OK;

Would need printfs or something to determine the 'frame rate' though, since obviously it won't render the video.

My only real guess is that it has something to do with us using D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX on the destination textures we create to copy into. The d3d9 code just uses query objects for synchronization (which sucks), so we could try doing the d3d11 equivalent of that.
(In reply to Matt Woodrow (:mattwoodrow) from comment #44)
> Ok, performs awfully :)
Sorry, I hope my comment wasn't taken as insulting, just wanted to make sure that I was clear about how it was working. I know if I thought it was just a little stuttering or something I'd probably try looking at something small rather than what might be a big picture thing here.

> It does work reasonably well on my windows 10 machine though.
Hmm... what hardware is that running on?
I wonder if this could be something driver-level then. If I recall correctly, AMD GPUs and hardware decoding weren't exactly fun to work with around here... I wouldn't put it past them to be doing something odd for Windows 7.

> What differences do you mean exactly?
I just wasn't sure what could be different exactly, my knowledge of DirectX in general isn't very good so I thought a little background info would be helpful before I start poking around in this. I appreciate you taking the time to explain some of this! I'll go see what happens with skipping the color conversion.

By the way, while I got ya here, something else I stumbled across on MSDN: "In Windows 7, the maximum supported resolution is 1920 × 1088 pixels for both software and DXVA decoding."
Trying to play a 1440p video with the D3D9 DXVA code results in a ~5 second lockup (no frames presented) and then Firefox falls back to software decoding. Seems it might be better to go directly to software decoding at that resolution instead. Should I file a separate bug for this?
Couple more builds to try out if you have time:

https://treeherder.mozilla.org/#/jobs?repo=try&revision=09fb5fb066f1

https://treeherder.mozilla.org/#/jobs?repo=try&revision=74a42cc29fcd

(In reply to Kyle Repinski from comment #45)
> Hmm... what hardware is that running on?
> I wonder if this could be something driver-level then. If I recall
> correctly, AMD GPUs and hardware decoding weren't exactly fun to work with
> around here... I wouldn't put it past them to be doing something odd for
> Windows 7.

It's an NVIDIA Quadro (1000m iirc).

It's definitely believable that the behaviour is driver specific.

 
> By the way, while I got ya here, something else I stumbled across on MSDN:
> "In Windows 7, the maximum supported resolution is 1920 × 1088 pixels for
> both software and DXVA decoding."
> Trying to play a 1440p video with the D3D9 DXVA code results in a ~5 second
> lockup (no frames presented) and then Firefox falls back to software
> decoding. Seems it might be better to go directly to software decoding at
> that resolution instead. Should I file a separate bug for this?

How are you determining that it's falling back to software? Generally speaking, switching decoders should be really fast.

The MSDN article suggests that software decoding wouldn't work either.
(In reply to Matt Woodrow (:mattwoodrow) from comment #46)
> Couple more builds to try out if you have time:
> 
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=09fb5fb066f1
> 
> https://treeherder.mozilla.org/#/jobs?repo=try&revision=74a42cc29fcd

No noticeable change. If I had to guess I'd say the first was slightly better than the previous builds I'd tried, and the second was slightly worse, but this is probably just placebo or something. I think I finally got the kinks worked out for doing local builds here, I'm going to see if a Win64 build behaves any differently.

> (In reply to Kyle Repinski from comment #45)
> > Hmm... what hardware is that running on?
> > I wonder if this could be something driver-level then. If I recall
> > correctly, AMD GPUs and hardware decoding weren't exactly fun to work with
> > around here... I wouldn't put it past them to be doing something odd for
> > Windows 7.
> 
> It's an NVIDIA Quadro (1000m iirc).
> 
> It's definitely believable that the behaviour is driver specific.

Shame I don't have an Intel or Nvidia system to test this on, it seems like this should be working fine to me too. Hopefully somebody else can give it a try with at least one of those two.

> > By the way, while I got ya here, something else I stumbled across on MSDN:
> > "In Windows 7, the maximum supported resolution is 1920 × 1088 pixels for
> > both software and DXVA decoding."
> > Trying to play a 1440p video with the D3D9 DXVA code results in a ~5 second
> > lockup (no frames presented) and then Firefox falls back to software
> > decoding. Seems it might be better to go directly to software decoding at
> > that resolution instead. Should I file a separate bug for this?
> 
> How are you determining that it's falling back to software? Generally
> speaking, switching decoders should be really fast.
> 
> The MSDN article suggests that software decoding wouldn't work either.

Well I guess I could be doing something a little more scientific to test this, but here's the behavior I'm observing:
1. Play 1080p YouTube video. Observe CPU usage of plugin-container.exe is hovering around 10%.
2. Change quality of same video to 1440p. Observe video decoding essentially freeze while stream advances the next 5 seconds, then when it picks back up the CPU usage of plugin-container.exe is juggling around between 60 and 80%.
3. Change quality back down to 1080p. Observe CPU usage of plugin-container.exe drop slightly, but still remain above 50%.
That seems to me like it's starting off with hardware decoding for 1080p, then determines hardware decoding isn't happening at 1440p and falls back to software, and since hardware decoding had already taken a nosedive it continues to use software decoding when going back down to 1080p again.
(In reply to Kyle Repinski from comment #47) 
> No noticeable change. If I had to guess I'd say the first was slightly
> better than the previous builds I'd tried, and the second was slightly
> worse, but this is probably just placebo or something. I think I finally got
> the kinks worked out for doing local builds here, I'm going to see if a
> Win64 build behaves any differently.

Well that's interesting. Basically rules out my theory on the keyed mutexes being an issue.

I guess we're back to trying to figure out if it's the colour conversion, or the decoding.

If you want to give this a go, it's probably worth grabbing the top changeset from the first link. That has code to create an ID3D11Query, which I think we'd want to leave in even if we remove the colour conversion code. That way we're actually blocking for the decode to complete, and should have useful timings.

Enabling the layers.acceleration.draw-fps should show fps numbers (the leftmost one) which will be useful for doing comparisons.


> 
> Shame I don't have an Intel or Nvidia system to test this on, it seems like
> this should be working fine to me too. Hopefully somebody else can give it a
> try with at least one of those two.

I'll be in an office next week, will try it out on a few machines there.


> Well I guess I could be doing something a little more scientific to test
> this, but here's the behavior I'm observing:
> 1. Play 1080p YouTube video. Observe CPU usage of plugin-container.exe is
> hovering around 10%.
> 2. Change quality of same video to 1440p. Observe video decoding essentially
> freeze while stream advances the next 5 seconds, then when it picks back up
> the CPU usage of plugin-container.exe is juggling around between 60 and 80%.
> 3. Change quality back down to 1080p. Observe CPU usage of
> plugin-container.exe drop slightly, but still remain above 50%.
> That seems to me like it's starting off with hardware decoding for 1080p,
> then determines hardware decoding isn't happening at 1440p and falls back to
> software, and since hardware decoding had already taken a nosedive it
> continues to use software decoding when going back down to 1080p again.

Seems reasonable. Right clicking on the video should show if we've fallen back to Flash.

The graphics section of about:support should show an error if we decide the hardware decoding is failing and abandon it.
(In reply to Matt Woodrow (:mattwoodrow) from comment #48)
> (In reply to Kyle Repinski from comment #47) 
> > No noticeable change. If I had to guess I'd say the first was slightly
> > better than the previous builds I'd tried, and the second was slightly
> > worse, but this is probably just placebo or something. I think I finally got
> > the kinks worked out for doing local builds here, I'm going to see if a
> > Win64 build behaves any differently.
> 
> Well that's interesting. Basically rules out my theory on the keyed mutexes
> being an issue.
> 
> I guess we're back to trying to figure out if it's the colour conversion, or
> the decoding.

Well there is something going on with the colour conversion that I just noticed. A test video I just tried is letterboxed, and the black bars on the top and bottom are very noticeably gray on D3D11 decoding. Everything else has them showing up in near-black.
This is the video in question: https://www.youtube.com/watch?v=45fvwFunaKE


> If you want to give this a go, it's probably worth grabbing the top
> changeset from the first link. That has code to create an ID3D11Query, which
> I think we'd want to leave in even if we remove the colour conversion code.
> That way we're actually blocking for the decode to complete, and should have
> useful timings.

Will do, thanks.


> Enabling the layers.acceleration.draw-fps should show fps numbers (the
> leftmost one) which will be useful for doing comparisons.

That's actually showing the full video framerate. It seems to think that it's not dropping any frames.


> > Well I guess I could be doing something a little more scientific to test
> > this, but here's the behavior I'm observing:
> > 1. Play 1080p YouTube video. Observe CPU usage of plugin-container.exe is
> > hovering around 10%.
> > 2. Change quality of same video to 1440p. Observe video decoding essentially
> > freeze while stream advances the next 5 seconds, then when it picks back up
> > the CPU usage of plugin-container.exe is juggling around between 60 and 80%.
> > 3. Change quality back down to 1080p. Observe CPU usage of
> > plugin-container.exe drop slightly, but still remain above 50%.
> > That seems to me like it's starting off with hardware decoding for 1080p,
> > then determines hardware decoding isn't happening at 1440p and falls back to
> > software, and since hardware decoding had already taken a nosedive it
> > continues to use software decoding when going back down to 1080p again.
> 
> Seems reasonable. Right clicking on the video should show if we've fallen
> back to Flash.
> 
> The graphics section of about:support should show an error if we decide the
> hardware decoding is failing and abandon it.

about:support still says Hardware H264 Decoding is OK, and Flash is disabled so it's not falling back to that either, but it's definitely not hardware decoding. Codec does not appear to change either.
Is there some way that WMF could itself be falling back to software decoding without Firefox even realizing it?
(In reply to Kyle Repinski from comment #49)
 
> Well there is something going on with the colour conversion that I just
> noticed. A test video I just tried is letterboxed, and the black bars on the
> top and bottom are very noticeably gray on D3D11 decoding. Everything else
> has them showing up in near-black.
> This is the video in question: https://www.youtube.com/watch?v=45fvwFunaKE

That's likely just a missing call to VideoProcessorSetStreamColorSpace.

The video is probably using 16-235 (where 16 would mean black), but is being interpreted as 0-255 (so 16 becomes gray).

Worth fixing, but seems very unlikely to be related to the performance issues.


> 
> > Enabling the layers.acceleration.draw-fps should show fps numbers (the
> > leftmost one) which will be useful for doing comparisons.
> 
> That's actually showing the full video framerate. It seems to think that
> it's not dropping any frames.

Well that's interesting. We shouldn't be compositing unless we really think we have a new video frame available.

Is anything else on the page animated? Youtube often has a 'loading' spinner for the comments section, scrolling down so that the comments load should give you a stable state (fps should drop to 0 with the video paused). The controls being visible will mess things up too obviously.

> 
> about:support still says Hardware H264 Decoding is OK, and Flash is disabled
> so it's not falling back to that either, but it's definitely not hardware
> decoding. Codec does not appear to change either.
> Is there some way that WMF could itself be falling back to software decoding
> without Firefox even realizing it?

Yeah, it could be I guess. The WMF Transform we use supports both hardware and software and we can't really tell which it's doing internally. The massive delay seems weird though.

We already attempt to detect what resolutions the hardware supports and don't enable the hardware flag for unsupported resolutions.
I have enough of this and installed Windows 10. MS Edge does not know what GPU features blacklist is and always use hardware acceleration in all cases.
Some people around here don't seem to understand the problem they created.
This whole feature has been removed via bug 1257013, so I am closing this as a duplicate.
Status: UNCONFIRMED → RESOLVED
Closed: 8 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: